Image pickup control apparatus, image pickup apparatus, control method for image pickup control apparatus, and non-transitory computer readable medium

ABSTRACT

To reduce power consumption in an image pickup apparatus that captures a plurality of pieces of image data. 
     An image pickup apparatus includes a signal processing unit and a control unit. The signal processing unit executes, in accordance with a predetermined control signal, either compound-eye processing for synthesizing a plurality of pieces of image data by carrying out signal processing on each of the plurality of pieces of image data or monocular processing for carrying out the signal processing on any one of the plurality of pieces of image data. The control unit supplies the predetermined control signal to the signal processing unit and causes one of the compound-eye processing and the monocular processing to be switched to the other one of the compound-eye processing and the monocular processing, on a basis of a result of a comparison between a measured predetermined physical amount and a predetermined threshold value.

CROSS REFERENCES TO RELATED APPLICATIONS

The present Application is a Continuation Application of U.S. patentapplication Ser. No. 16/648,997 filed Mar. 19, 2020, which is a 371National Stage Entry of International Application No.:PCT/JP2018/029570, filed on Aug. 7, 2018, which in turn claims priorityfrom Japanese Application No. 2017-184263, filed on Sep. 26, 2017, theentire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present technology relates to an image pickup control apparatus, animage pickup apparatus, a control method for the image pickup controlapparatus, and a non-transitory computer readable medium morespecifically, to an image pickup control apparatus including a pluralityof solid-state image pickup devices, an image pickup apparatus, acontrol method for the image pickup control apparatus, and anon-transitory computer readable medium.

BACKGROUND ART

From the past, for simultaneously capturing a plurality of pieces ofimage data, a compound-eye image pickup apparatus including a pluralityof solid-state image pickup devices has been used. For example, there isproposed an image pickup apparatus that captures color image data usinga solid-state image pickup device including a color filter, and capturesmonochrome image data using a solid-state image pickup device includingno color filter (see, for example, Patent Literature 1). Here, sinceincident light is not attenuated by a color filter, the monochrome imagedata becomes that much brighter than the color image data. This imagepickup apparatus simultaneously captures and synthesizes those colorimage data and monochrome image pickup data.

CITATION LIST Patent Literature

-   [PTL 1]    JP2011-239260

SUMMARY Technical Problem

According to the technology of the past described above, by synthesizingrelatively-bright monochrome image data with color image data, colorimage data that is brighter than that in a case where the image data isnot synthesized can be generated. However, since a processing amount ofsynthesis processing for synthesizing a plurality of pieces of imagedata is larger than that in a case of processing one piece of imagedata, there is a problem that power consumption of the image pickupapparatus is increased as compared to the case where the image data isnot synthesized. As the power consumption increases, there is a fearthat a lowering speed of a remaining battery amount of the image pickupapparatus and a heat generation amount of the image pickup apparatuswill increase, so it is desirable to reduce power consumption.

The present technology has been made in view of the circumstances asdescribed above and aims at reducing power consumption in an imagepickup apparatus that captures a plurality of pieces of image data.

Solution to Problem

The present technology has been made to solve the problems describedabove, and according to a first aspect of the present technology, thereis provided an image pickup control apparatus including: a signalprocessing unit that executes, in accordance with a predeterminedcontrol signal, either compound-eye processing for synthesizing aplurality of pieces of image data by carrying out signal processing oneach of the plurality of pieces of image data or monocular processingfor carrying out the signal processing on any one of the plurality ofpieces of image data; and a control unit that supplies the predeterminedcontrol signal to the signal processing unit and causes one of thecompound-eye processing and the monocular processing to be switched tothe other one of the compound-eye processing and the monocularprocessing, on a basis of a result of a comparison between a measuredpredetermined physical amount and a predetermined threshold value, and acontrol method therefor. Accordingly, an operation that the processingof the signal processing unit is switched from one of the compound-eyeprocessing and the monocular processing to the other one of thecompound-eye processing and the monocular processing in accordance withthe physical amount can be obtained.

Further, in this first aspect, the predetermined physical amount may bea physical amount that fluctuates in accordance with an image pickupenvironment. Accordingly, an operation that the processing of the signalprocessing unit is switched from one of the compound-eye processing andthe monocular processing to the other one of the compound-eye processingand the monocular processing in accordance with the image pickupenvironment can be obtained.

Further, in this first aspect, the image pickup control apparatus mayfurther include a temperature sensor that measures a temperature as thepredetermined physical amount, and the control unit may cause thecompound-eye processing to be switched to the monocular processing in acase where the temperature is higher than the predetermined thresholdvalue. Accordingly, an operation that the processing of the signalprocessing unit is switched from the compound-eye processing to themonocular processing in a case where the temperature is higher than thethreshold value can be obtained.

Further, in this first aspect, the signal processing unit may includeprocessing of measuring a light amount of incident light as thepredetermined physical amount and acquiring the light amount as aphotometric amount, and the control unit may cause the compound-eyeprocessing to be switched to the monocular processing in a case wherethe photometric amount is larger than the predetermined threshold value.Accordingly, an operation that the processing of the signal processingunit is switched from the compound-eye processing to the monocularprocessing in a case where the photometric amount is larger than thethreshold value can be obtained.

Further, in this first aspect, the signal processing unit may includeprocessing of measuring a movement amount of a subject as thepredetermined physical amount, and the control unit may cause themonocular processing to be switched to the compound-eye processing in acase where the movement amount is larger than the predeterminedthreshold value. Accordingly, an operation that the processing of thesignal processing unit is switched from the monocular processing to thecompound-eye processing in a case where the movement amount of thesubject is larger than the threshold value can be obtained.

Further, in this first aspect, the plurality of pieces of image data maybe images captured in synchronization with predetermined synchronizationsignals, and the signal processing unit may switch one of thecompound-eye processing and the monocular processing to the other one ofthe compound-eye processing and the monocular processing after executingtransition processing for performing a synthesis while changing asynthesis ratio every time a predetermined cycle passes. Accordingly, anoperation that the processing is switched after gradually changing thesynthesis ratio can be obtained.

Further, according to a second aspect of the present technology, thereis provided an image pickup control apparatus including: a synthesisprocessing unit that synthesizes a plurality of pieces of image data andoutputs the data as synthetic image data; a transmission unit thatexecutes, in accordance with a predetermined control signal, eithersingle-channel transmission processing for transmitting the syntheticimage data together with a single piece of image data as any one of theplurality of pieces of image data via a single communication channel ormultiple-channel transmission processing for respectively transmittingthe single piece of image data and the synthetic image data via mutuallydifferent communication channels; and a control unit that supplies thepredetermined control signal to the signal processing unit and causesone of the single-channel transmission processing and themultiple-channel transmission processing to be switched to the other oneof the single-channel transmission processing and the multiple-channeltransmission processing, on a basis of a result of a comparison betweena measured predetermined physical amount and a predetermined thresholdvalue. Accordingly, an operation that one of the single-channeltransmission processing and the multiple-channel transmission processingis switched to the other one of the single-channel transmissionprocessing and the multiple-channel transmission processing inaccordance with the physical amount can be obtained.

Further, in this second aspect, the image pickup apparatus may furtherinclude a scaling processing unit that reduces any one of the pluralityof pieces of image data, and the control unit may control the scalingprocessing unit to change a reduction ratio on the basis of the resultof the comparison between the predetermined physical amount and thepredetermined threshold value. Accordingly, an operation that thereduction ratio is changed in accordance with the physical amount can beobtained.

Further, in this second aspect, the image pickup apparatus may furtherinclude a synchronization control unit that supplies a predeterminedsynchronization signal, each of the plurality of pieces of image datamay be image data that has been captured in synchronization with thepredetermined synchronization signal, and the control unit may controlthe synchronization control unit to change a frequency of thepredetermined synchronization signal on the basis of the result of thecomparison between the predetermined physical amount and thepredetermined threshold value. Accordingly, an operation that thefrequency of the synchronization signal is changed in accordance withthe physical amount can be obtained.

Further, according to a third aspect of the present technology, there isprovided an image pickup apparatus including: a camera module thatcaptures a plurality of pieces of image data; a signal processing unitthat executes, in accordance with a predetermined control signal, eithercompound-eye processing for synthesizing the plurality of pieces ofimage data by carrying out signal processing on each of the plurality ofpieces of image data or monocular processing for carrying out the signalprocessing on any one of the plurality of pieces of image data; and acontrol unit that supplies the predetermined control signal to thesignal processing unit and causes one of the compound-eye processing andthe monocular processing to be switched to the other one of thecompound-eye processing and the monocular processing, on a basis of aresult of a comparison between a measured predetermined physical amountand a predetermined threshold value. Accordingly, an operation that theprocessing on the image data from the camera module is switched from oneof the compound-eye processing and the monocular processing to the otherone of the compound-eye processing and the monocular processing inaccordance with the physical amount can be obtained.

Advantageous Effects of Invention

According to the present technology, it is possible to achieve anexcellent effect that power consumption can be reduced in an imagepickup apparatus that captures a plurality of pieces of image data. Itshould be noted that the effects described herein are not necessarilylimited, and any of the effects described in the present disclosure maybe obtained.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1

A block diagram showing a configuration example of an image pickupapparatus according to a first embodiment of the present technology.

FIG. 2

Block diagrams showing a configuration example of a camera moduleaccording to the first embodiment of the present technology.

FIG. 3

An example of a cross-sectional diagram of a solid-state image pickupdevice according to the first embodiment of the present technology.

FIG. 4

A block diagram showing a configuration example of a signal processingunit according to the first embodiment of the present technology.

FIG. 5

A block diagram showing a configuration example of a color image signalprocessing unit according to the first embodiment of the presenttechnology.

FIG. 6

A block diagram showing a configuration example of a synthesisprocessing unit according to the first embodiment of the presenttechnology.

FIG. 7

A timing chart showing an example of an operation of the image pickupapparatus according to the first embodiment of the present technology.

FIG. 8

A block diagram showing an example of a state of the image pickupapparatus in a case where synthesis processing is not carried out in thefirst embodiment of the present technology.

FIG. 9

A flowchart showing an example of an operation of the image pickupapparatus according to the first embodiment of the present technology.

FIG. 10

A block diagram showing a configuration example of an image pickupapparatus according to a first modified example of the first embodimentof the present technology.

FIG. 11

A block diagram showing a configuration example of a signal processingunit according to the first modified example of the first embodiment ofthe present technology.

FIG. 12

A block diagram showing a configuration example of a color image signalprocessing unit according to a second modified example of the firstembodiment of the present technology.

FIG. 13

A block diagram showing a configuration example of a signal processingunit according to a second embodiment of the present technology.

FIG. 14

A block diagram showing a configuration example of an applicationprocessor according to the second embodiment of the present technology.

FIG. 15

A block diagram showing a configuration example of an image pickupapparatus according to a modified example of the second embodiment ofthe present technology.

FIG. 16

A block diagram showing a schematic configuration example of a vehiclecontrol system.

FIG. 17

An explanatory diagram showing an example of a setting position of animage pickup unit.

DESCRIPTION OF EMBODIMENTS

Hereinafter, modes for embodying the present technology (hereinafter,referred to as embodiments) will be described. Descriptions will begiven in the following order.

1. First embodiment (example of switch from compound-eye processing tomonocular processing when temperature is higher than threshold value)

2. Second embodiment (example of switch from 2 channels to 1 channelwhen temperature is higher than threshold value)

3. Application example to mobile object

1. First Embodiment

“Configuration Example of Image Pickup Apparatus”

FIG. 1 is a block diagram showing a configuration example of an imagepickup apparatus 100 according to a first embodiment of the presenttechnology. This image pickup apparatus 100 captures image data andincludes camera modules 110 and 130, an image pickup control apparatus200, and an application processor 150. As the image pickup apparatus100, a smartphone, an action cam, an IoT (Internet of Things) camera, anin-vehicle camera, and the like are assumed.

The camera modules 110 and 130 capture image data in synchronizationwith a vertical synchronization signal VSYNC having a predeterminedfrequency (30 hertz, etc.). These camera modules 110 and 130simultaneously capture image data in synchronization with the samevertical synchronization signal VSYNC. Then, the camera module 110supplies the image data to the image pickup control apparatus 200 via asignal line 119, and the camera module 130 supplies the image data tothe image pickup control apparatus 200 via a signal line 139.

Also, the camera module 110 operates or stops in accordance with anenable signal ENc1 from the image pickup control apparatus 200, and thecamera module 130 operates or stops in accordance with an enable signalENc2 from the image pickup control apparatus 200. The enable signal ENc1is transmitted via a signal line 217, and the enable signal ENc2 istransmitted via a signal line 218.

The image pickup control apparatus 200 controls an image pickupoperation of the image pickup apparatus 100 and includes a control unit210, a temperature sensor 220, and a signal processing unit 240.

The temperature sensor 220 measures a temperature of the image pickupapparatus 100. This temperature sensor 220 supplies temperature dataindicating a measurement value to the control unit 210 via a signal line229.

The signal processing unit 240 carries out predetermined signalprocessing on two pieces of image data from the camera modules 110 and130. Under control of the control unit 210, this signal processing unit240 executes compound-eye processing or monocular processing. Here, thecompound-eye processing is processing of carrying out signal processingon each of the two pieces of image data and synthesizing them. Further,the monocular processing is processing of carrying out signal processingon only one of the two pieces of image data. The signal processing unit240 supplies the processed image data to the application processor 150via a signal line 209.

Further, the signal processing unit 240 controls an exposure amount ofthe camera modules 110 and 130 and a position of a focus lens.

The application processor 150 executes various types of image processingsuch as object recognition processing and filter processing on imagedata from the signal processing unit 240. For example, this applicationprocessor 150 records processed image data in a nonvolatile memory (notshown) or the like. Further, for example, the application processor 150supplies the processed image data to a display apparatus (not shown) fordisplay.

The control unit 210 controls the camera modules 110 and 130 and thesignal processing unit 240 on the basis of a result of a comparisonbetween a measured predetermined physical amount and a predeterminedthreshold value. It is desirable for the physical amount to be measuredto fluctuate in accordance with an image pickup environment. Atemperature is used as such a physical amount, for example. When apredetermined application for performing image pickup is executed, thecontrol unit 210 supplies the enable signals ENc1 and ENc2 to the signalprocessing unit 240 to operate both of the camera modules 110 and 130.In addition, at that time, the control unit 210 supplies enable signalsENs1, ENs2, and ENs3 to the signal processing unit 240 to cause thesignal processing unit 240 to execute the compound-eye processing.Details of the enable signals ENs1 to ENs3 will be described later.These enable signals ENs1 to ENs3 are transmitted via a signal line 219.It should be noted that the enable signals ENs1, ENs2, and ENs3 areexamples of a control signal described in the scope of claims.

Then, the control unit 210 monitors the temperature and determineswhether or not the temperature is higher than a predetermined upperlimit threshold value. When the temperature becomes higher than theupper limit threshold value, the control unit 210 supplies the enablesignals ENs1, ENs2, and ENs3 to the signal processing unit 240 tocontrol the signal processing unit 240 to start a switch from thecompound-eye processing to the monocular processing. Then, when theswitch is completed, the control unit 210 supplies the enable signalENc1 or ENc2 to the signal processing unit 240 to stop one of the cameramodules 110 and 130.

In this way, when the temperature becomes higher than the upper limitthreshold value, one of the camera modules 110 and 130 is stopped and aswitch is made to the monocular processing, with the result that powerconsumption is reduced, and a temperature rise is suppressed.

Further, after switching to the monocular processing, the control unit210 determines whether or not the temperature is lower than apredetermined lower limit threshold value. When the temperature becomeslower than the lower limit threshold value, the control unit 210controls the signal processing unit 240 by the enable signals ENs1,ENs2, and ENs3 to start a switch from the monocular processing to thecompound-eye processing. Then, when the switch is completed, the controlunit 210 causes both of the camera modules 110 and 130 to operate by theenable signals ENc1 and ENc2.

“Configuration Example of Camera Module”

FIG. 2 are block diagrams showing a configuration example of the cameramodules 110 and 130 according to the first embodiment of the presenttechnology. “a” in the figures is a block diagram showing aconfiguration example of the camera module 110, and “b” in the figuresis a block diagram showing a configuration example of the camera module130.

The camera module 110 includes an optical system 111 and a solid-stateimage pickup device 120. The optical system 111 is constituted of aplurality of optical components including a diaphragm and a focus lens,and collects incident light and guides it to the solid-state imagepickup device 120. The solid-state image pickup device 120 capturescolor image data including color information and luminance informationin synchronization with the vertical synchronization signal VSYNC.

The signal processing unit 240 controls an aperture value of the opticalsystem 111 and an exposure time of the solid-state image pickup device120 to set an exposure amount to an appropriate value. In addition, thesignal processing unit 240 controls a position of the focus lens of theoptical system 111 for focus.

Further, the camera module 130 includes an optical system 131 and asolid-state image pickup device 140. A configuration of the opticalsystem 131 is similar to that of the optical system 111. The solid-stateimage pickup device 140 captures monochrome image data including onlyluminance information in synchronization with the verticalsynchronization signal VSYNC. The signal processing unit 240 alsocontrols an exposure amount and focal point of the camera module 130.

It should be noted that although the optical system 111 and solid-stateimage pickup device 120 and the optical system 131 and solid-state imagepickup device 140 are arranged in separate camera modules, all of themmay be arranged in a single camera module.

FIG. 3 are an example of cross-sectional diagrams of the solid-stateimage pickup devices 120 and 140 according to the first embodiment ofthe present technology. “a” in the figures is an example of thecross-sectional diagram of the solid-state image pickup device 120, and“b” in the figures is an example of the cross-sectional diagram of thesolid-state image pickup device 140.

On a light reception surface of the solid state image pickup device 120,a microlens 121 is provided for each pixel. A color filter 122 isprovided below each of the microlenses 121 with a direction that isdirected from the microlens 121 toward a focal point being a downwarddirection. A light reception device 123 is arranged below each of thecolor filters 122. This color filter 122 transmits light having any of ared wavelength, a green wavelength, and a blue wavelength. By performinga photoelectric conversion on those transmitted light, color image datain which respective pixel signals of R (Red), G (Green) and B (Blue) arearranged in a two-dimensional lattice is generated. A wiring layer 124is provided below the light reception devices 123. With the wiring layer124 being a front surface, the solid-state image pickup device 120 inwhich light is irradiated onto a back surface with respect to the frontsurface as described above, is called a back-irradiation-typesolid-state image pickup device.

On a light reception surface of the solid-state image pickup device 140,a microlens 141 are provided for each pixel. A light reception device143 is provided below each of the microlenses 141. A wiring layer 144 isprovided below the light reception devices 143. Since the solid-stateimage pickup device 140 does not include a color filter, monochromeimage data is captured by the solid-state image pickup device 140. Inaddition, since incident light is not attenuated by the color filter inthe solid-state image pickup device 140, monochrome image data that isbrighter than color image data can be obtained under light sources ofthe same light amount.

It should be noted that although the back-irradiation-type solid-stateimage pickup device is used as the solid-state image pickup devices 120and 140, a front-irradiation-type solid-state image pickup device canalso be used.

“Configuration Example of Signal Processing Unit”

FIG. 4 is a block diagram showing a configuration example of the signalprocessing unit 240 according to the first embodiment of the presenttechnology. This signal processing unit 240 includes a color imagesignal processing unit 241, a monochrome image signal processing unit247, a synthesis processing unit 250, and a transmission unit 248.

The color image signal processing unit 241 executes predetermined signalprocessing on color image data from the camera module 110. This colorimage signal processing unit 241 supplies the processed color image datato the synthesis processing unit 250. Further, the color image signalprocessing unit 241 operates or stops in accordance with the enablesignal ENs1 from the control unit 210.

The monochrome image signal processing unit 247 executes predeterminedsignal processing on monochrome image data from the camera module 130.This monochrome image signal processing unit 247 supplies the processedmonochrome image data to the synthesis processing unit 250. Further, themonochrome image signal processing unit 247 operates or stops inaccordance with the enable signal ENs2 from the control unit 210.

The synthesis processing unit 250 synthesizes color image data andmonochrome image data that have been subjected to signal processing.This synthesis processing unit 250 synthesizes the color image data andthe monochrome image data in synchronization with the verticalsynchronization signal VSYNC to generate synthetic image data. Then, thesynthesis processing unit 250 changes a synthesis ratio in accordancewith the enable signal ENs3 from the control unit 210. Here, the enablesignal ENs3 is a signal for instructing either a start or an end of asynthesis. A method of changing a synthesis ratio will be describedlater. The synthesis processing unit 250 supplies the processed imagedata to the transmission unit 248 as output image data.

The transmission unit 248 transmits output image data. This transmissionunit 248 transmits output image data to the application processor 150 inaccordance with an MIPI (Mobile Industry Processor Interface) standard,for example.

FIG. 5 is a block diagram showing a configuration example of the colorimage signal processing unit 241 according to the first embodiment ofthe present technology. This color image signal processing unit 241includes a de-mosaic processing unit 242, a detection unit 243, anexposure control unit 244, and a focus control unit 245.

The de-mosaic processing unit 242 carries out de-mosaic processing forinterpolating, for each pixel, insufficient color information fromsurrounding pixels, on the color image data. By the de-mosaicprocessing, color image data including R, G, and B signals for eachpixel is generated. This de-mosaic processing unit 242 supplies thecolor image data subjected to the de-mosaic processing to the synthesisprocessing unit 250. It should be noted that in addition to thede-mosaic processing, the color image signal processing unit 241 mayfurther execute other processing such as filter processing and defectivepixel interpolation processing.

The detection unit 243 measures a light amount of incident light fromthe color image data and acquires it as a photometric amount. Forexample, the photometric amount can be obtained as a detection value bya center-weighted photometry method or a spot photometry method. Thedetection unit 243 supplies the detection value (photometric amount) tothe exposure control unit 244.

The exposure control unit 244 controls an exposure amount to anappropriate value on the basis of the detection value. This exposurecontrol unit 244 controls an aperture value and exposure time of thecamera module 110 to set the exposure amount to an appropriate value.

The focus control unit 245 focuses the camera module 110. This focuscontrol unit 245 detects a position in focus using, for example, animage plane phase difference detection method or a contrast detectionmethod, for example, and moves the focus lens of the camera module 110to that position.

The configuration of the monochrome image signal processing unit 247 issimilar to that of the color image signal processing unit 241 exceptthat the de-mosaic processing is not executed.

“Configuration Example of Synthesis Processing Unit”

FIG. 6 is a block diagram showing a configuration example of thesynthesis processing unit 250 according to the first embodiment of thepresent technology. This synthesis processing unit 250 includes apositioning processing unit 251, a YC conversion unit 252, a luminancesignal synthesis unit 253, an RGB conversion unit 254, and a switch 255.

The positioning processing unit 251 positions the color image data andthe monochrome image data. This positioning processing unit 251 correctsa positional deviation between the two pieces of image data, that is dueto an attachment error or disparity of the camera modules 110 and 130,by a parallel movement, a rotation, or the like. The positioningprocessing unit 251 supplies the positioned color image data to the YCconversion unit 252 and supplies the monochrome image data to theluminance signal synthesis unit 253.

The YC conversion unit 252 converts, for each pixel in the color imagedata, R, G, and B signals into a luminance signal Y1 and colordifference signals Cb and Cr by the following expressions.Y1=0.257R+0.504G+0.098B+16Cb=−0.148R−0.291G+0.439B+128Cr=0.439R−0.368G−0.071B+128

The YC conversion unit 252 supplies the luminance signal Y1 to theluminance signal synthesis unit 253 and supplies the color differencesignals Cb and Cr to the RGB conversion unit 254.

The luminance signal synthesis unit 253 synthesizes the luminance signalY1 of the color image data and a luminance signal Y2 of the monochromeimage data for each pixel in accordance with the following expression.Y3=Y1*(A/100)+Y2*(1−A)/100In the above expression, A represents a synthesis ratio in a percentage,and Y3 represents a luminance signal obtained after the synthesis.

The luminance signal synthesis unit 253 supplies the luminance signal Y3obtained after the synthesis to the RGB conversion unit 254.

The RGB conversion unit 254 converts the luminance signal Y3 and thecolor difference signals Cr and Cb into R, G, and B signals for eachpixel by the following expressions. The RGB conversion unit 254 suppliesthe converted image data to the switch 255.R=1.164(Y3−16)−1.596(Cr−128)G=1.164(Y3−16)−0.391(Cb−128)−0.813(Cr−128)B=1.164(Y3−16)−2.018(Cb−128)

The switch 255 selects either the image data from the RGB conversionunit 254 or the color image data from the color image signal processingunit 241 under control of the luminance signal synthesis unit 253. In aninitial state, the image data from the RGB conversion unit 254 isselected. This switch 255 supplies the selected image data to thetransmission unit 248 as output image data.

As a synthesis start is instructed by the enable signal ENs3, theluminance signal synthesis unit 253 sets an initial value (e.g., “0”) ofa synthesis ratio A and starts the synthesis. Then, when a synthesis endis instructed by the enable signal ENs3 after the start of thesynthesis, the luminance signal synthesis unit 253 executes transitionprocessing for increasing the synthesis ratio A every time a framecycle, which is a cycle of the vertical synchronization signal VSYNC,passes. For example, in a case where the synthesis ratio A isincremented by “10” percent (%) every frame cycle, the synthesis ratio Abecomes “100” percent (%) at 10 frame cycles, and the transitionprocessing ends. At the time the transition processing ends, theluminance signal synthesis unit 253 controls the switch 255 to selectcolor image data from the color image signal processing unit 241. Byexecuting this transition processing, an abrupt change in brightness inthe output image data can be suppressed.

Then, when a synthesis start is instructed by the enable signal ENs3after ending the synthesis, the luminance signal synthesis unit 253executes transition processing for decreasing the synthesis ratio Aevery time the frame cycle passes. In addition, the luminance signalsynthesis unit 253 controls the switch 255 to select the image data fromthe RGB conversion unit 254.

It should be noted that although the synthesis processing unit 250synthesizes the monochrome image data and the color image data, acombination of the image data to be synthesized is not limited to themonochrome image data and the color image data. For example, thesynthesis processing unit 250 may synthesize two pieces of color imagedata. In addition, although the synthesis processing unit 250 performsthe synthesis while separating the luminance signal and the colordifference signals, the synthesis may be performed without separatingthem.

FIG. 7 is a timing chart showing an example of an operation of the imagepickup apparatus 100 according to the first embodiment of the presenttechnology. Before image pickup starts, all of the enable signals ENc1,ENc2, ENs1, ENs2, and ENs3 are set to, for example, disable.

When an application or the like for performing image pickup is executedat a timing T0, the control unit 210 sets all of the enable signalsENc1, ENc2, ENs1, ENs2, and ENs3 to enable. As a result, the cameramodules 110 and 130 start operating. In addition, the signal processingunit 240 executes compound-eye processing and outputs synthetic imagedata.

Due to the operations of the camera modules 110 and 130 and an increaseof a processing amount of the signal processing unit 240, powerconsumption and a heat generation amount of the image pickup apparatus100 increase, and a temperature of the image pickup apparatus 100gradually increases. Then, at a timing T1, the temperature becomeshigher than an upper limit threshold value Th1. Here, in an electronicapparatus, in order to prevent breakage of circuits due to heat, asafety circuit for stopping operations of the circuits when atemperature of the apparatus becomes higher than a certain stopthreshold value is incorporated in many cases. As the upper limitthreshold value Th1, a value lower than such a stop threshold value isset.

At the timing T1, the control unit 210 sets the enable signal ENs3 todisable on the basis of a comparison result between the temperature andthe upper limit threshold value Th1, to thus start a switch from thecompound-eye processing to the monocular processing.

In accordance with the enable signal ENs3, the synthesis processing unit250 executes the transition processing for gradually changing thesynthesis ratio. This transition processing ends after a certain timepasses. For example, in a case where the synthesis ratio A isincremented by “10” percent (%) every frame cycle, the synthesis ratio Abecomes “100” percent (%) at 10 frame cycles, and the transitionprocessing ends.

At a timing T2 where the transition processing ends, the control unit210 stops the camera module 130 and the monochrome image signalprocessing unit 247 in accordance with the enable signal ENc2 and theenable signal ENs2. By stopping the monochrome image signal processingunit 247, the signal processing unit 240 starts the monocular processingand outputs color image data.

By stopping the camera module 130 and switching from the compound-eyeprocessing to the monocular processing, the power consumption and heatgeneration amount of the image pickup apparatus 100 decrease, and thetemperature of the image pickup apparatus 100 gradually decreases.

Then, when the temperature becomes lower than a lower limit thresholdvalue Th2 at a timing T3, the control unit 210 sets the enable signalsENs2 and ENs3 to enable to start a switch from the monocular processingto the compound-eye processing.

The monochrome image signal processing unit 247 starts operating inaccordance with the enable signal ENs2, and the synthesis processingunit 250 executes the transition processing for gradually changing thesynthesis ratio in accordance with the enable signal ENs3.

It should be noted that although the signal processing unit 240 executesthe transition processing first and then executes the processing towhich the switch has been made (monocular processing or compound-eyeprocessing) when a switch is instructed by the enable signal, it is alsopossible to execute those processing without executing the transitionprocessing.

FIG. 8 is a block diagram showing an example of a state of the imagepickup apparatus in a case where the synthesis processing according tothe first embodiment of the present technology is not carried out. Whenthe temperature becomes higher than the upper limit threshold value Th1,the control unit 210 disables the enable signals ENc2, ENs2, and ENs3.As a result, the camera module 130 and the monochrome image signalprocessing unit 247 stop operating, and the synthesis processing unit250 stops the synthesis processing. Therefore, the power consumption andheat generation amount of the image pickup apparatus 100 are reduced.

For example, when the temperature of the image pickup apparatus 100rises to exceed the stop threshold value during recording, there is apossibility that a part of the circuits will automatically stop forprotection of the circuits of the image pickup apparatus 100 and for thesafety of a user, and the recording will thus be interrupted. However,when the temperature becomes higher than the upper limit threshold valueTh1 which is lower than that stop threshold value, the image pickupapparatus 100 switches from the compound-eye processing to the monocularprocessing to reduce the power consumption and heat generation amount.Therefore, the temperature rise is suppressed, and the recording can becontinued.

“Operational Example of Image Pickup Apparatus”

FIG. 9 is a flowchart showing an example of an operation of the imagepickup apparatus 100 according to the first embodiment of the presenttechnology. This operation is started when an application of theprocessing for performing image pickup is executed, for example.

The image pickup apparatus 100 starts capturing color image data andmonochrome image data (Step S901). Then, the image pickup apparatus 100carries out the synthesis processing for synthesizing the color imagedata and the monochrome image data (Step S902).

The image pickup apparatus 100 determines whether or not the temperatureis higher than the upper limit threshold value Th1 (Step S903). In acase where the temperature is equal to or lower than the upper limitthreshold value Th1 (Step S903: No), the image pickup apparatus 100executes Step S902 and the subsequent steps again.

On the other hand, in a case where the temperature is higher than theupper limit threshold value Th1 (Step S903: Yes), the image pickupapparatus 100 executes the transition processing (Step S904) andexecutes the monocular processing (Step S905).

Then, the image pickup apparatus 100 determines whether or not thetemperature is lower than the lower limit threshold value Th2 (StepS906). In a case where the temperature is equal to or higher than thelower limit threshold value Th2 (Step S906: No), the image pickupapparatus 100 executes Step S905 and the subsequent steps again.

On the other hand, in a case where the temperature is lower than thelower limit threshold value Th2 (Step S906: Yes), the image pickupapparatus 100 executes the transition processing (Step S907) andrepetitively executes Step S902 and the subsequent steps.

In this way, in the first embodiment of the present technology, when thetemperature becomes higher than the upper limit threshold value Th1, thecontrol unit 210 controls the signal processing unit 240 to switch fromthe compound-eye processing to the monocular processing, so it becomespossible to reduce the processing amount of the signal processing unit240 and reduce power consumption thereof. For various different examplesof physical amounts, the physical amount can be determined according toa voltage value that varies in correspondence with the physical amount.For example, a temperature sensor may output a voltage value that variesin accordance with the sensed temperature, a voltage value may vary inaccordance with changes in a remaining amount of battery capacity, etc.

First Modified Example

In the first embodiment described above, when the temperature becomeshigher than the upper limit threshold value Th1, the control unit 210causes the compound-eye processing to be switched to the monocularprocessing to thus reduce power consumption. However, by the method ofswitching in accordance with a temperature rise, power consumption maynot be sufficiently reduced. For example, in a case where an outsidetemperature of the image pickup apparatus 100 is extremely low, theimage pickup apparatus 100 is cooled by outside air so that thetemperature becomes equal to or lower than the threshold value, and thusthe switch to the monocular processing may not be executed. The controlunit 210 according to a first modified example of the first embodimentis different from that of the first embodiment in that a switch from thecompound-eye processing to the monocular processing is performed inaccordance with a physical amount other than the temperature.

FIG. 10 is a block diagram showing a configuration example of the imagepickup apparatus 100 according to the first modified example of thefirst embodiment of the present technology. This image pickup apparatus100 according to the first modified example of the first embodiment isdifferent from that of the first embodiment in that a statisticsprocessing unit 230 is provided in place of the temperature sensor 220.

The statistics processing unit 230 obtains a statistical amount (averagevalue, total value, etc.) of detection values from the signal processingunit 240. The signal processing unit 240 supplies a detection value ofthe color image data and a detection value of the monochrome image datato the statistics processing unit 230 via a signal line 238. Then, thestatistics processing unit 230 obtains a statistical amount of thosedetection values and supplies it to the control unit 210 via a signalline 239.

The control unit 210 compares the statistical amount from the statisticsprocessing unit 230 with a threshold value and makes a switch betweenthe monocular processing and the compound-eye processing on the basis ofthe comparison result. This statistical amount indicates a photometricamount obtained from the two pieces of image data. For example, when thestatistical amount (photometric amount) becomes larger than an upperlimit threshold value, the control unit 210 causes the compound-eyeprocessing to be switched to the monocular processing. Accordingly, in acase where the image pickup environment is bright, there is no need toimprove luminance by the synthesis, so the monocular processing isexecuted, and power consumption is reduced. On the other hand, in a casewhere the photometric amount becomes smaller than a lower limitthreshold value, the control unit 210 causes the monocular processing tobe switched to the compound-eye processing.

FIG. 11 is a block diagram showing a configuration example of the signalprocessing unit 240 according to the first modified example of the firstembodiment of the present technology. In this first modified example ofthe first embodiment, the color image signal processing unit 241 and themonochrome image signal processing unit 247 supply detection values tothe statistics processing unit 230.

As described above, in the first modified example of the firstembodiment of the present technology, when the photometric amountbecomes larger than the threshold value, the control unit 210 causes thecompound-eye processing to be switched to the monocular processing, soit becomes possible to reduce the processing amount of the signalprocessing unit 240 in accordance with the photometric amount and reducepower consumption thereof.

Second Modified Example

In the first modified example of the first embodiment described above,when the photometric amount becomes larger than the threshold value, thecontrol unit 210 makes a switch from the compound-eye processing to themonocular processing to thus reduce power consumption. However, by themethod of switching in accordance with the photometric amount, there isa possibility that the power consumption will not be sufficientlyreduced. For example, in a case where the image pickup environment ofthe image pickup apparatus 100 is dark, the photometric amount maybecome smaller than the threshold value, and a switch to the monocularprocessing may not be executed. The control unit 210 according to thissecond modified example of the first embodiment is different from thatof the first modified example of the first embodiment in that the switchfrom the compound-eye processing to the monocular processing isperformed in accordance with a physical amount other than thephotometric amount.

FIG. 12 is a block diagram showing a configuration example of the colorimage signal processing unit 241 according to the second modifiedexample of the first embodiment of the present technology. This colorimage signal processing unit 241 according to the second modifiedexample of the first embodiment is different from that of the firstmodified example of the first embodiment in that it further includes amovement amount measurement unit 246.

The movement amount measurement unit 246 measures a movement amount of asubject on the basis of a plurality of pieces of color image datacaptured in time series. This movement amount measurement unit 246temporarily stores a predetermined number of pieces of color image data,for example, and detects a moving object using an inter-frame differencemethod, a background difference method, or the like. Then, the movementamount measurement unit 246 calculates, as the movement amount, a lengthof a vector from a representative position (center of gravity, etc.) ofa moving body detected in one of the two pieces of color image datacaptured at different timings to a representative position of the movingbody detected in the other one of the two pieces of color image data.The movement amount measurement unit 246 supplies the measured movementamount to the statistics processing unit 230.

Also, the monochrome image signal processing unit 247 measures amovement amount and supplies it to the statistics processing unit 230.The statistics processing unit 230 obtains a statistical amount (averagevalue etc.) of the movement amount and supplies it to the control unit210. For example, when the statistical amount is equal to or larger thana threshold value, the control unit 210 causes the monocular processingto be switched to the compound-eye processing. On the other hand, whenthe statistical amount becomes smaller than the threshold value, thecontrol unit 210 causes the compound-eye processing to be switched tothe monocular processing. Accordingly, in a case where there is nomovement of the subject, the synthesis processing for improvingluminance can be stopped, and thus the power consumption can be reduced.

It should be noted that although the image pickup apparatus 100 makes aswitch between the compound-eye processing and the monocular processingon the basis of a comparison result between the movement amount and thethreshold value, it is also possible to make a switch on the basis of acomparison result between a remaining battery amount and a thresholdvalue. In this case, a remaining battery amount measurement unit thatmeasures a remaining battery amount only needs to be added so that thecontrol unit 210 controls the signal processing unit 240 to switch fromthe compound-eye processing to the monocular processing when theremaining battery amount becomes smaller than the threshold value, forexample.

In this way, in the second modified example of the first embodiment ofthe present technology, when the movement amount becomes smaller thanthe threshold value, the control unit 210 causes the compound-eyeprocessing to be switched to the monocular processing, so it becomespossible to reduce the processing amount of the signal processing unit240 in accordance with the movement amount, and thus power consumptionthereof can be reduced.

2. Second Embodiment

In the first embodiment described above, when the temperature becomeshigher than the upper limit threshold value Th1, the control unit 210makes a switch from the compound-eye processing to the monocularprocessing to reduce power consumption. However, since the synthesisprocessing for improving luminance is stopped when switched to themonocular processing, there is a possibility that image quality of imagedata will be lowered due to insufficient brightness. The image pickupapparatus 100 according to the second embodiment is different from thatof the first embodiment in that power consumption of an interface isreduced while executing synthesis processing at a time the temperaturebecomes higher than the upper limit threshold value Th1.

FIG. 13 is a block diagram showing a configuration example of the signalprocessing unit 240 according to the second embodiment of the presenttechnology. This signal processing unit 240 according to the secondembodiment is different from that of the first embodiment in that itfurther includes a scaling processing unit 249.

The scaling processing unit 249 reduces an image size of monochromeimage data that has been subjected to signal processing by apredetermined reduction ratio. Here, a reduction ratio R is expressed bythe following expression, for example.R=(image size after reduction)/(image size before reduction)

This scaling processing unit 249 supplies image data obtained after thereduction to the transmission unit 248 as reduced image data. Further, aswitch signal SW1 from the control unit 210 is input to the scalingprocessing unit 249. This switch signal SW1 is a signal that instructs aswitch from one of an initial value and a target value regarding thereduction ratio to the other one of the values. Further, as the targetvalue, a value that is smaller than the initial value and with whichreduced image data can be superimposed on synthetic image data to betransmitted by 1 channel is set. It should be noted that the initialvalue may be “1” that is a value at which reduction is not executed.

Further, a switch signal SW2 from the control unit 210 is input to thetransmission unit 248 of the second embodiment. This switch signal SW2is a signal that instructs a switch from one of 1 channel and 2 channelsto the other one of the 1 channel and the 2 channels regarding thenumber of communication channels used for transmitting image data. Here,as the communication channel, for example, a virtual channel conformingto an MIPI standard is used. In a case of transmitting by 1 channel, thetransmission unit 248 superimposes reduced image data on synthetic imagedata and transmits it.

The control unit 210 of the second embodiment sets the enable signalsENs1 to ENs3 to enable at a time of starting image pickup and causes thescaling processing unit 249 to perform reduction by an initial valuereduction ratio by the switch signal SW1. Further, the control unit 210sets the communication channel to be 2 channels by the switch signal SW2and causes the synthetic image data and the reduced image data to betransmitted. These pieces of image data are transmitted via mutuallydifferent communication channels.

Then, when the temperature becomes higher than the upper limit thresholdvalue Th1, the control unit 210 instructs the scaling processing unit249 to switch from the initial value to the target value regarding thereduction ratio by the switch signal SW1. The scaling processing unit249 executes transition processing for decreasing the reduction ratio tothe target value stepwise for each frame cycle. Further, the controlunit 210 causes the number of communication channels to be switched from2 channels to 1 channel by the switch signal SW2. Meanwhile, in thesecond embodiment, even if the temperature rises, the synthesisprocessing is continuously executed.

Then, when the temperature becomes lower than the lower limit thresholdvalue Th2, the control unit 210 instructs the scaling processing unit249 to switch the reduction ratio to the initial value by the switchsignal SW1. The scaling processing unit 249 executes transitionprocessing for increasing the reduction ratio to the initial valuestepwise for each frame cycle. Further, the control unit 210 causes thenumber of communication channels to be switched from 1 channel to 2channels by the switch signal SW2.

FIG. 14 is a block diagram showing a configuration example of theapplication processor 150 according to the second embodiment of thepresent technology. This application processor 150 includes a receptionunit 151, an image recognition unit 152, and a memory 153.

The reception unit 151 receives image data from the image pickup controlapparatus 200. This reception unit 151 receives the reduced image dataand the synthetic image data via a communication channel of 1 channel or2 channels. Then, the reception unit 151 supplies the reduced image datato the image recognition unit 152 and causes the memory 153 to store thesynthetic image data.

The image recognition unit 152 executes image recognition processingsuch as face recognition and object recognition on the reduced imagedata. The recognition result is used to judge whether or not to store(i.e., record) the synthetic image data in a nonvolatile memory, forexample.

The memory 153 stores the synthetic image data. A DRAM (Dynamic RAM) isused as the memory 153, for example. The synthetic image data stored inthe memory 153 is supplied to a display apparatus or the nonvolatilememory, for example.

As described above, by reducing the number of communication channels,power consumption of the interface including the transmission unit 248and the reception unit 151 can be reduced.

It should be noted that although the image pickup apparatus 100 switchesthe number of communication channels on the basis of a comparison resultbetween the temperature and the threshold value, it is also possible tomake a switch on the basis of a comparison result between a physicalamount other than the temperature and a threshold value. For example,the image pickup apparatus 100 can use physical amounts such as aphotometric amount, a movement amount of a subject, and a remainingbattery amount for determining a switch.

As described above, in the second embodiment of the present technology,when the temperature becomes higher than the upper limit threshold valueTh1, the control unit 210 reduces the number of communication channels,so the power consumption of the interface can be reduced.

Modified Example

In the second embodiment described above, when the temperature becomeshigher than the upper limit threshold value Th1, the control unit 210changes the reduction ratio to a small value with which image data canbe transmitted even when the number of communication channels isreduced. However, if the reduction ratio becomes small, an image size ofthe reduced image data becomes small, and recognition accuracy in theimage recognition is lowered. The image pickup apparatus 100 accordingto a modified example of the second embodiment is different from that ofthe first embodiment in that, when the temperature becomes higher thanthe upper limit threshold value Th1, a frame rate is lowered withoutchanging the reduction ratio.

FIG. 15 is a block diagram showing a configuration example of the imagepickup apparatus 100 according to the modified example of the secondembodiment of the present technology. This image pickup apparatus 100according to the modified example of the second embodiment is differentfrom that of the second embodiment in that a synchronization controlunit 260 is additionally provided in the image pickup control apparatus200.

The synchronization control unit 260 supplies the verticalsynchronization signal VSYNC to the camera modules 110 and 130 and thesignal processing unit 240. A switch signal SW3 from the control unit210 is input to this synchronization control unit 260 via a signal line269. This switch signal SW3 is a signal that instructs a switch from oneof an initial value and a target value regarding a frequency (i.e.,frame rate) of the vertical synchronization signal VSYNC to the otherone of the values. As the target value, a value that is lower than theinitial value and with which reduced image data can be superimposed onsynthetic image data to be transmitted by 1 channel is set.

In addition, the switch signal SW1 for switching the reduction ratio isnot input to the signal processing unit 240 according to the modifiedexample of the second embodiment.

When the temperature becomes higher than the upper limit threshold valueTh1, the control unit 210 instructs a switch of the frame rate from theinitial value to the target value by the switch signal SW3. Then, whenthe temperature becomes lower than the lower limit threshold value Th2,the control unit 210 instructs a switch of the frame rate to the initialvalue by the switch signal SW3. Meanwhile, in the modified example ofthe second embodiment, the reduction ratio is not changed.

In this way, in the modified example of the second embodiment of thepresent technology, when the temperature becomes higher than the upperlimit threshold value Th1, the control unit 210 reduces the number ofcommunication channels to reduce the frame rate, so it becomes possibleto reduce the power consumption of the interface without changing thereduction ratio.

3. Application Example to Mobile Body

The technology according to the present disclosure (present technology)is applicable to various products. For example, the technology accordingto the present disclosure may be realized as an apparatus mounted on anytype of moving bodies, such as an automobile, an electric vehicle, ahybrid electric vehicle, a motorcycle, a bicycle, a personal mobility,an airplane, a drone, a ship, and a robot.

FIG. 16 is a block diagram showing a schematic configuration example ofa vehicle control system as an example of a mobile body control systemto which the technology according to the present disclosure isapplicable.

A vehicle control system. 12000 includes a plurality of electroniccontrol units connected via a communication network 12001. In theexample shown in FIG. 16 , the vehicle control system 12000 includes adriving system controller unit 12010, a body system controller unit12020, a vehicle exterior information detector unit 12030, an in-vehicleinformation detector unit 12040, and an integrated controller unit12050. Further, as a functional configuration of the integratedcontroller unit 12050, a microcomputer 12051, an audio image output unit12052, and an in-vehicle network I/F (interface) 12053 are illustrated.

The driving system controller unit 12010 controls an operation of anapparatus related to a driving system of a vehicle in accordance withvarious programs. For example, the driving system controller unit 12010functions as a control apparatus for a driving force generationapparatus for generating a driving force of the vehicle, such as aninternal combustion engine and a driving motor, a driving forcetransmission mechanism for transmitting a driving force to the wheels, asteering mechanism that adjusts a rudder angle of the vehicle, a brakingapparatus for generating a braking force of the vehicle, and the like.

The body system controller unit 12020 controls operations of variousapparatuses equipped in a vehicle body in accordance with variousprograms. For example, the body system controller unit 12020 functionsas a control apparatus for a keyless entry system, a smart key system, apower window apparatus, and various lamps such as headlamps, backinglight, brake light, turn indicators, and fog lamps. In this case, radiowaves transmitted from a mobile device that substitutes for a key orsignals of various switches can be input to the body system controllerunit 12020. The body system controller unit 12020 receives an input ofthese radio waves or signals and controls a door lock apparatus, thepower window apparatus, the lamps, and the like of the vehicle.

The vehicle exterior information detector unit 12030 detects externalinformation of the vehicle on which the vehicle control system 12000 ismounted. For example, an image pickup unit 12031 is connected to thevehicle exterior information detector unit 12030. The vehicle exteriorinformation detector unit 12030 causes the image pickup unit 12031 tocapture an image outside the vehicle and receives the captured image.The vehicle exterior information detector unit 12030 may carry outobject detection processing or distance detection processing withrespect to a person, a car, an obstacle, a sign, characters on a roadsurface, and the like on the basis of the received image.

The image pickup unit 12031 is a light sensor that receives light andoutputs an electric signal corresponding to a light reception amount ofthat light. The image pickup unit 12031 is capable of outputting anelectric signal as an image or outputting it as distance measurementinformation. Moreover, the light received by the image pickup unit 12031may either be visible light or invisible light such as infrared light.

The in-vehicle information detector unit 12040 detects informationinside the vehicle. A driver's-state detection unit 12041 that detects astate of a driver is connected to the in-vehicle information detectorunit 12040, for example. The driver's-state detection unit 12041includes, for example, a camera that captures the driver, and thein-vehicle information detector unit 12040 may calculate a degree offatigue or a degree of concentration of the driver or determine whetheror not the driver is dozing off on the basis of detection informationinput from the driver's-state detection unit 12041.

The microcomputer 12051 is capable of calculating a control target valueof the driving force generation apparatus, the steering mechanism, orthe braking apparatus on the basis of exterior information or in-vehicleinformation acquired by the vehicle exterior information detector unit12030 or the in-vehicle information detector unit 12040, and outputtinga control command to the driving system controller unit 12010. Forexample, the microcomputer 12051 is capable of performing cooperativecontrol that aims at realizing a function of ADAS (Advanced DriverAssistance System) that includes collision avoidance or impactmitigation of a vehicle, follow-up running based on a distance betweenvehicles, running while maintaining a vehicle speed, vehicle collisionwarning, vehicle lane deviation warning, and the like.

Further, the microcomputer 12051 controls the driving force generationapparatus, the steering mechanism, the braking apparatus, or the like onthe basis of peripheral information of the vehicle acquired by thevehicle exterior information detector unit 12030 or the in-vehicleinformation detector unit 12040, so as to be capable of performingcooperative control that aims at realizing automatic driving that allowsa vehicle to run autonomously without depending on an operation of thedriver, and the like.

Furthermore, the microcomputer 12051 is capable of outputting a controlcommand to the body system controller unit 12020 on the basis of thevehicle exterior information acquired by the vehicle exteriorinformation detector unit 12030. For example, the microcomputer 12051 iscapable of controlling the headlamps in accordance with a position of apreceding vehicle or an oncoming vehicle that has been detected by thevehicle exterior information detector unit 12030, and performingcooperative control that aims at realizing antiglare by switching a highbeam to a low beam, or the like.

The audio image output unit 12052 transmits an output signal of at leastone of audio and an image to an output apparatus capable of visually oraudibly notifying a passenger of the vehicle or outside the vehicle, ofinformation. In the example shown in FIG. 16 , an audio speaker 12061, adisplay unit 12062, and an instrument panel 12063 are exemplified as theoutput apparatus. The display unit 12062 may include at least one of anon-board display and a head-up display, for example.

FIG. 17 is a diagram showing an example of a setting position of theimage pickup unit 12031.

In FIG. 17 , image pickup units 12101, 12102, 12103, 12104, and 12105are included as the image pickup unit 12031.

For example, the image pickup units 12101, 12102, 12103, 12104, and12105 are provided at positions corresponding to a front nose, sidemirrors, rear bumper, back door, upper portion of a front window in acabin, and the like of a vehicle 12100. The image pickup unit 12101provided at the front nose and the image pickup unit 12105 provided atthe upper portion of the front window in the cabin mainly acquire animage in front of the vehicle 12100. The image pickup units 12102 and12103 provided at the side mirrors mainly acquire images of sides of thevehicle 12100. The image pickup unit 12104 provided at the rear bumperor the back door mainly acquires an image behind the vehicle 12100. Theimage pickup unit 12105 provided at the upper portion of the frontwindow in the cabin is mainly used for detecting preceding vehicles,pedestrians, obstacles, traffic signals, traffic signs, lanes, and thelike.

It should be noted that FIG. 17 shows an example of image pickup rangesof the image pickup units 12101 to 12104. An image pickup range 12111indicates an image pickup range of the image pickup unit 12101 providedat the front nose, image pickup ranges 12112 and 12113 indicate imagepickup ranges of the image pickup units 12102 and 12103 respectivelyprovided at the side mirrors, and an image pickup range 12114 indicatesan image pickup range of the image pickup unit 12104 provided at therear bumper or the back door. For example, by causing image datacaptured by the image pickup units 12101 to 12104 to overlap oneanother, an overhead view image of the vehicle 12100 viewed from abovecan be obtained.

At least one of the image pickup units 12101 to 12104 may include afunction of acquiring distance information. For example, at least one ofthe image pickup units 12101 to 12104 may be a stereo camera including aplurality of image pickup devices, or may be an image pickup deviceincluding pixels for a phase difference detection.

For example, the microcomputer 12051 can calculate a distance to eachthree-dimensional object within the image pickup ranges 12111 to 12114and a temporal change (relative speed with respect to vehicle 12100) ofthe distances on the basis of distance information obtained from theimage pickup units 12101 to 12104, to thus extract a three-dimensionalobject that is closest to the vehicle 12100 on a traveling path and istraveling at a predetermined speed (e.g., 0 km/h or more) insubstantially the same direction as the vehicle 12100 in particular, asa preceding vehicle. Furthermore, the microcomputer 12051 can set aninter-vehicle distance to be secured in advance behind the precedingvehicle so as to perform automatic brake control (including follow-upstop control), automatic acceleration control (including follow-up startcontrol), and the like. In this way, it is possible to performcooperative control that aims at realizing automatic driving in whichthe vehicle runs autonomously without depending on an operation of thedriver, and the like.

For example, on the basis of the distance information obtained from theimage pickup units 12101 to 12104, the microcomputer 12051 can extractthree-dimensional object data related to the three-dimensional objectwhile categorizing it as a two-wheeled vehicle, a standard-sizedvehicle, a large-sized vehicle, a pedestrian, or other three-dimensionalobjects such as a utility pole, and use it for automatic avoidance ofobstacles. For example, the microcomputer 12051 identifies an obstaclein a periphery of the vehicle 12100 as an obstacle visible by the driverof the vehicle 12100 or an obstacle that is difficult to be seen. Then,the microcomputer 12051 determines a collision risk that indicates adegree of risk of a collision with each obstacle, and when the collisionrisk is equal to or larger than a setting value and thus there is apossibility of a collision, a warning is output to the driver via theaudio speaker 12061 or the display unit 12062, or forced deceleration oravoidance steering is performed via the driving system controller unit12010, so that driving assistance for avoiding a collision can beperformed.

At least one of the image pickup units 12101 to 12104 may be an infraredcamera that detects infrared rays. For example, the microcomputer 12051can recognize a pedestrian by judging whether or not there is apedestrian in images captured by the image pickup units 12101 to 12104.Such a recognition of a pedestrian is carried out by, for example, aprocedure of extracting feature points in images captured by the imagepickup units 12101 to 12104 as the infrared camera and a procedure ofcarrying out pattern matching processing on a series of feature pointsindicating an outline of an object to judge whether or not there is apedestrian. When the microcomputer 12051 judges that a pedestrian existsin the captured images of the image pickup units 12101 to 12104 andrecognizes a pedestrian, the audio image output unit 12052 controls thedisplay unit 12062 to superimpose a rectangular outline for emphasis onthe recognized pedestrian during display. Alternatively, the audio imageoutput unit 12052 may control the display unit 12062 to display an iconor the like indicating a pedestrian at a desired position.

Heretofore, an example of the vehicle control system to which thetechnology according to the present disclosure is applicable has beendescribed. The technology according to the present disclosure isapplicable to the image pickup unit 12031 out of the configurationdescribed above. Specifically, the image pickup apparatus 100 shown inFIG. 1 is applicable to the image pickup unit 12031 shown in FIG. 16 .By applying the technology according to the present disclosure to theimage pickup unit 12031, power consumption of the image pickup unit12031 can be reduced, and thus consumption of an on-board battery can besuppressed.

It should be noted that the embodiments described above are examples forembodying the present technology, and the matters in the embodiments andthe specific matters of the invention in the scope of claimsrespectively have correspondence relationships. Similarly, the specificmatters of the invention in the scope of claims and the matters in theembodiments of the present technology having the same names as thespecific matters of the invention in the scope of claims respectivelyhave correspondence relationships. However, the present technology isnot limited to the embodiments described above and can be embodied bybeing variously modified without departing from the gist of the presenttechnology.

Further, the processing procedures described in the embodimentsdescribed above may be regarded as a method including the series ofprocedures, or may be regarded as a program for causing a computer toexecute the series of procedures or a recording medium that stores sucha program. Examples of this recording medium include a CD (CompactDisc), an MD (Mini Disc), a DVD (Digital Versatile Disc), a memory card,a Blu-ray (registered trademark) disc, and the like.

It should be noted that the effects described in the specification aremere examples and should not be limited thereto, and other effects mayalso be obtained.

It should be noted that the present technology can also take thefollowing configurations.

(1) An image pickup control apparatus, including:

-   -   a signal processing unit that executes, in accordance with a        predetermined control signal, either compound-eye processing for        synthesizing a plurality of pieces of image    -   data by carrying out signal processing on each of the plurality        of pieces of image data or monocular processing for carrying out        the signal processing on any one of the plurality of pieces of        image data; and    -   a control unit that supplies the predetermined control signal to        the signal processing unit and causes one of the compound-eye        processing and the monocular processing to be switched to the        other one of the compound-eye processing and the monocular        processing, on a basis of a result of a comparison between a        measured predetermined physical amount and a predetermined        threshold value.

(2) The image pickup control apparatus according to (1), in which

-   -   the predetermined physical amount is a physical amount that        fluctuates in accordance with an image pickup environment.

(3) The image pickup control apparatus according to (2), furtherincluding

-   -   a temperature sensor that measures a temperature as the        predetermined physical amount,    -   in which    -   the control unit causes the compound-eye processing to be        switched to the monocular processing in a case where the        temperature is higher than the predetermined threshold value.

(4) The image pickup control apparatus according to (2), in which

-   -   the signal processing unit includes processing of measuring a        light amount of incident light as the predetermined physical        amount and acquiring the light amount as a photometric amount,        and    -   the control unit causes the compound-eye processing to be        switched to the monocular processing in a case where the        photometric amount is larger than the predetermined threshold        value.

(5) The image pickup control apparatus according to (1), in which

-   -   the signal processing unit includes processing of measuring a        movement amount of a subject as the predetermined physical        amount, and    -   the control unit causes the monocular processing to be switched        to the compound-eye processing in a case where the movement        amount is larger than the predetermined threshold value.

(6) The image pickup control apparatus according to any one of (1) to(5), in which

-   -   the plurality of pieces of image data are images captured in        synchronization with predetermined synchronization signals, and    -   the signal processing unit switches one of the compound-eye        processing and the monocular processing to the other one of the        compound-eye processing and the monocular processing after        executing transition processing for performing a synthesis while        changing a synthesis ratio every time a predetermined cycle        passes.

(7) An image pickup control apparatus, including:

-   -   a synthesis processing unit that synthesizes a plurality of        pieces of image data and outputs the data as synthetic image        data;    -   a transmission unit that executes, in accordance with a        predetermined control signal, either single-channel transmission        processing for transmitting the synthetic image data together        with a single piece of image data as any one of the plurality of        pieces of image data via a single communication channel or        multiple-channel transmission processing for respectively        transmitting the single piece of image data and the synthetic        image data via mutually different communication channels; and    -   a control unit that supplies the predetermined control signal to        the signal processing unit and causes one of the single-channel        transmission processing and the multiple-channel transmission        processing to be switched to the other one of the single-channel        transmission processing and the multiple-channel transmission        processing, on a basis of a result of a comparison between a        measured predetermined physical amount and a predetermined        threshold value.

(8) The image pickup apparatus according to (7), further including

-   -   a scaling processing unit that reduces any one of the plurality        of pieces of image data,    -   in which    -   the control unit controls the scaling processing unit to change        a reduction ratio on the basis of the result of the comparison        between the predetermined physical amount and the predetermined        threshold value.

(9) The image pickup apparatus according to (7), further including

-   -   a synchronization control unit that supplies a predetermined        synchronization signal,    -   in which    -   each of the plurality of pieces of image data is image data that        has been captured in synchronization with the predetermined        synchronization signal, and    -   the control unit controls the synchronization control unit to        change a frequency of the predetermined synchronization signal        on the basis of the result of the comparison between the        predetermined physical amount and the predetermined threshold        value.

(10) An image pickup apparatus, including:

-   -   a camera module that captures a plurality of pieces of image        data;    -   a signal processing unit that executes, in accordance with a        predetermined control signal, either compound-eye processing for        synthesizing the plurality of pieces of image data by carrying        out signal processing on each of the plurality of pieces of        image data or monocular processing for carrying out the signal        processing on any one of the plurality of pieces of image data;        and    -   a control unit that supplies the predetermined control signal to        the signal processing unit and causes one of the compound-eye        processing and the monocular processing to be switched to the        other one of the compound-eye processing and the monocular        processing, on a basis of a result of a comparison between a        measured predetermined physical amount and a predetermined        threshold value.

(11) A control method for an image pickup control apparatus, including:

-   -   a comparison step of comparing a measured predetermined physical        amount and a predetermined threshold value and supplying a        predetermined control signal to a signal processing unit on a        basis of a result of the comparison; and    -   a switch step of switching, in accordance with the predetermined        control signal, one of compound-eye processing for synthesizing        a plurality of pieces of image data by carrying out signal        processing on each of the plurality of pieces of image data and        monocular processing for carrying out the signal processing on        any one of the plurality of pieces of image data to the other        one of the compound-eye processing and the monocular processing.

(1A) An image pickup control apparatus, comprising:

-   -   a signal processor configured to execute compound processing        that synthesizes a plurality of pieces of image data and        monocular processing that processes any one of the plurality of        pieces of image data; and    -   a controller configured to perform a comparison of a physical        amount to a threshold value and cause the signal processor to        execute either the compound processing or the monocular        processing on a basis of the comparison.

(2A) The image pickup control apparatus according to (1A), wherein

-   -   the physical amount is a measured physical condition that        fluctuates in an image pickup environment.

(3A) The image pickup control apparatus according to (2A), furthercomprising:

-   -   a temperature sensor that measures a temperature as the physical        amount,    -   wherein    -   the controller causes the signal processor to switch the        compound processing to the monocular processing in a case where        the temperature is higher than the threshold value.

(4A) The image pickup control apparatus according to (2A), wherein

-   -   the controller determines an amount of incident light as the        physical amount, and    -   causes the signal processor to switch the compound processing to        the monocular processing in a case where the amount of incident        light is larger than the threshold value.

(5A) The image pickup control apparatus according to (1A), wherein

-   -   the controller determines a movement amount of a subject as the        physical amount, and    -   causes the signal processor to switch the monocular processing        to the compound processing in a case where the movement amount        is larger than the threshold value.

(6A) The image pickup control apparatus according to (1A), wherein

-   -   the plurality of pieces of image data are images captured in        synchronization with predetermined synchronization signals, and    -   the signal processor switches one of the compound processing and        the monocular processing to the other one of the compound        processing and the monocular processing after executing        transition processing for performing a synthesis while changing        a synthesis rate every time a predetermined cycle passes.

(7A) The image pickup control apparatus according to (2A), wherein

-   -   the controller causes the signal processor to gradually switch        the compound processing to the monocular processing once it is        determined that the physical amount is higher than the threshold        value.

(8A) The image pickup control apparatus according to (2A), wherein

-   -   after the controller causes the signal processor to switch to        the monocular processing upon determination that the physical        amount is higher than the threshold value, the controller causes        the monocular processing to continue the monocular processing        until the physical amount decreases to a second threshold value        that is lower than the threshold value.

(9A) The image pickup control apparatus according to (2A), wherein

-   -   the controller causes the signal processor to terminate both the        compound processing and the monocular processing when the        physical amount increases to a third threshold value that is        higher than the threshold value.

(10A) The image pickup control apparatus according to (1A), wherein

-   -   the plurality of pieces of image data comprise a first piece of        image data captured by a first camera module and a second piece        of image data captured by a second camera module.

(11A) The image pickup control apparatus according to (1A), wherein

-   -   the physical amount is a remaining amount of capacity of a        battery that supplies power for the image pickup control        apparatus.

(12A) The image pickup control apparatus according to (1A), wherein

-   -   the physical amount is determined according to a voltage value        that varies in correspondence with the physical amount.

(13A) An image pickup apparatus, comprising:

-   -   at least one lens configured to capture a plurality of pieces of        image data; and    -   a signal processor configured to execute compound processing        that synthesizes the plurality of pieces of image data or        monocular processing that carries out processing on any one of        the plurality of pieces of image data; and    -   a controller configured to cause the signal processor to execute        either the compound processing or the monocular processing, on a        basis of a result of a comparison between a physical amount and        a threshold value.

(14A) An image pickup control apparatus, comprising:

-   -   a signal processor configured to synthesize a plurality of        pieces of image data to produce synthesized image data;    -   a transmission unit configured to execute either single-channel        transmission processing for transmitting the synthetized image        data together with any one piece of image data of the plurality        of pieces of image data via a single communication channel or        multiple-channel transmission processing for respectively        transmitting the any one piece of image data and the synthetized        image data via mutually different communication channels; and    -   a controller configured to cause the signal processor to execute        either the single-channel transmission processing or the        multiple-channel transmission processing, on a basis of a result        of a comparison between a physical amount and a threshold value.

(15A) The image pickup apparatus according to (14A), further comprising:

-   -   scaling processing circuitry that reduces any one of the        plurality of pieces of image data,    -   wherein    -   the controller controls the scaling processing circuitry to        change a reduction ratio on the basis of the result of the        comparison between the physical amount and the threshold value.

(16A) The image pickup apparatus according to (14A), further comprising:

-   -   synchronization control circuitry that supplies a predetermined        synchronization signal,    -   wherein    -   each of the plurality of pieces of image data is image data that        has been captured in synchronization with the predetermined        synchronization signal, and    -   the controller controls the synchronization control circuitry to        change a frequency of the predetermined synchronization signal        on the basis of the result of the comparison between the        physical amount and the threshold value.

(17A) A control method for an image pickup control apparatus,comprising:

-   -   controlling a signal processor that is configured to execute        compound processing that synthesizes a plurality of pieces of        image data and monocular processing that processes any one of        the plurality of pieces of image data; and    -   performing a comparison of a physical amount to a threshold        value and causing the signal processor to execute either the        compound processing or the monocular processing on a basis of        the comparison.

(18A) A non-transitory computer readable medium storing program code forprocessing image data, the program code being executable by a processorto perform operations comprising:

-   -   controlling a signal processor that is configured to execute        compound processing that synthesizes a plurality of pieces of        image data and monocular processing that processes any one of        the plurality of pieces of image data; and    -   performing a comparison of a physical amount to a threshold        value and causing the signal processor to execute either the        compound processing or the monocular processing on a basis of        the comparison.

REFERENCE SIGNS LIST

-   100 image pickup apparatus-   110, 130 camera module-   111, 131 optical system-   120, 140 solid-state image pickup device-   121, 141 microlens-   122 color filter-   123, 143 light reception device-   124, 144 wiring layer-   150 application processor-   151 reception unit-   152 image recognition unit-   153 memory-   200 image pickup control apparatus-   210 control unit-   220 temperature sensor-   230 statistics processing unit-   240 signal processing unit-   241 color image signal processing unit-   242 de-mosaic processing unit-   243 detection unit-   244 exposure control unit-   245 focus control unit-   246 movement amount measurement unit-   247 monochrome image signal processing unit-   248 transmission unit-   249 scaling processing unit-   250 synthesis processing unit-   251 positioning processing unit-   252 YC conversion unit-   253 luminance signal synthesis unit-   254 RGB conversion unit-   255 switch-   260 synchronization control unit-   12031 image pickup unit

The invention claimed is:
 1. An image pickup control apparatus,comprising: a controller configured to control a signal processor toswitch between compound processing that synthesizes a plurality ofpieces of image data and monocular processing that processes any one ofthe plurality of pieces of image data, in response to comparisons of ameasured value to a first threshold value and a second threshold valuethat is lower than the first threshold value; and wherein after thecontroller causes the signal processor to continue to execute thecompound processing until the measured value is higher than the firstthreshold value, the controller causes the signal processor to return tothe compound processing from the monocular processing in response to themeasured value becoming lower than the second threshold value.
 2. Theimage pickup control apparatus according to claim 1, wherein afterdetermining that the measured value is higher than the first thresholdvalue, the controller causes the signal processor to enter a transitionprocessing followed by the monocular processing.
 3. The image pickupcontrol apparatus according to claim 2, wherein the transitionprocessing includes gradually changing a synthesis rate for synthesizingthe plurality of pieces of image data.
 4. The image pickup controlapparatus according to claim 1, wherein the measured value correspondsto a physical condition that fluctuates in an image pickup environment.5. The image pickup control apparatus according to claim 4, wherein thephysical condition is temperature.
 6. The image pickup control apparatusaccording to claim 4, wherein the physical condition is an amount ofincident light.
 7. The image pickup control apparatus according to claim4, wherein the physical condition is a movement amount of a subject. 8.The image pickup control apparatus according to claim 4, wherein thephysical condition is a remaining amount of capacity of a battery thatsupplies power for the image pickup control apparatus.
 9. The imagepickup control apparatus according to claim 1, wherein the controllercauses the signal processor to terminate both the compound processingand the monocular processing when the measured value increases to athird threshold value that is higher than the first threshold value. 10.An image pickup apparatus, comprising: at least one lens configured tocapture a plurality of pieces of image data; and a controller configuredto control signal processing to switch between compound processing thatsynthesizes the plurality of pieces of image data and monocularprocessing that processes any one of the plurality of pieces of imagedata, in response to comparisons of a measured value to a firstthreshold value and a second threshold value that is lower than thefirst threshold value, wherein after the controller causes the signalprocessing to continue to execute the compound processing until themeasured value is higher than the first threshold value, the controllercauses the signal processing to return to the compound processing fromthe monocular processing in response to the measured value becominglower than the second threshold value.
 11. The image pickup apparatusaccording to claim 10, wherein after determining that the measured valueis higher than the first threshold value, the controller causes thesignal processing to enter a transition processing followed by themonocular processing.
 12. The image pickup apparatus according to claim11, wherein the transition processing includes gradually changing asynthesis rate for synthesizing the plurality of pieces of image data.13. The image pickup apparatus according to claim 10, wherein themeasured value corresponds to a physical condition that fluctuates in animage pickup environment.
 14. The image pickup apparatus according toclaim 13, wherein the physical condition is temperature.
 15. Anon-transitory computer readable medium storing program code forprocessing image data, the program code being executable by a processorto perform operations comprising: controlling a signal processor toswitch between compound processing that synthesizes a plurality ofpieces of image data and monocular processing that processes any one ofthe plurality of pieces of image data, in response to comparisons of ameasured value to a first threshold value and a second threshold valuethat is lower than the first threshold value; and after controlling thesignal processor to continue to execute the compound processing untilthe measured value is higher than the first threshold value, causing thesignal processor to return to the compound processing from the monocularprocessing in response to the measured value becoming lower than thesecond threshold value.
 16. The non-transitory computer readable mediumaccording to claim 15, wherein the operations further comprise: afterdetermining that the measured value is higher than the first thresholdvalue, causing the signal processor to enter a transition processingfollowed by the monocular processing.
 17. The non-transitory computerreadable medium according to claim 16, wherein the transition processingincludes gradually changing a synthesis rate for synthesizing theplurality of pieces of image data.
 18. The non-transitory computerreadable medium according to claim 15, wherein the measured valuecorresponds to a physical condition that fluctuates in an image pickupenvironment.
 19. The non-transitory computer readable medium accordingto claim 18, wherein the physical condition is temperature.